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ABSTRACT 

In a series of four interrelated reports. Rand work 
to date for the Air Force on the devf^lopment of methodologies for 
designing programs of instruction is reported. This report, the first 
of the series, describes MODIA (A Method of Designing Instructional 
Alternatives) , a. comprehensive methodology for designing an 
instructional system. MODIA consists of a sequence of procedures and 
semiautomated tools (some developed and some in the planning stage) 
that allow a designer to examine many alternative structural 
approaches before he puts an actual system into use. A manual test of 
MODIA on an Air Force training course indicated that systematic, 
generalized methods can be developed for designing programs of 
instruction; these techniques encourage examination of alternatives;' 
and that a comprehensive^ systematic approach stimulates insights 
into the design of instruction that would otherwise not occur. 
(Author/RH) 
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PREFACE 



This is one of four interrelated reports describing Rand work for the Air Force 
to date on the development of methodologies for designi ng programs of instruction. 
The reports in the series are: 

R-1018-PR: An Overview of MODIA: A Method of Designing Instructional 

Alternatives for Air Force Training, Polly Carpenter. 
R-1019-PR, The MODIA Decision Process for Developing Strategies of Air 

Force Instruction, Polly Carpenter and Barbara Horner. 
R-1020-PR, The MODIA Questionnaire for Curriculum Analysis, Rudy 

Bretz. 

R-1021-PR, MODIA applied in the Design and Cost Analysis of an Innova- 
tive Air Force Course, Robert L. Petruschell and Polly Carpenter. 

The first of these provides an overview of the methodologies being developed; the 
second and third describe some of the major analytical tools used to provide inputs 
to the design process; and the last sets forth the results of a completed design cycle, 
parts of which were carried out manually, applied to a specific course in Air Force 
technical training. 

This work has been conducted under a Rand project entitled Analysis of Systems 
for Air Force Education and Training, Emphasis has been on the use of technology 
in designing instruction for formal technical training or for higher education, as at 
the Air Force Academy. The results will support the activities of the Director of 
Personnel Plans, Headquarters USAF; DCS/Technical Training and the Training 
Development Directorate, headquarters Air Training Command; and the Air Force 
Human Resources Laboratory, especially the Technical Training and Professional 
Education divisions. It will be of particular interest to those working on the Ad- 
vanced Instructional System. 

This report is part of a continuing Rand effort to apply systematic methods of 
analysis and synthesis to issues and problems in education and training. Related 
work in education, both military and civilian, concerns Air Force pilot training and 
management of the pilot force, evaluation of programs of compensatory education, 
design of information systems for local school districts, and other diverse concerns. 
A special bibliography of Rand work in education is available on request, 
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R- 1018 -PR AN OVERVIEW OF MODIA: A METHOD OF DESIGNING INSTRUCTIONAL 
ALTERNATIVES FOR AIR FORCE TRAINING, Polly Carpenter, 
Nwember 1972, Unclassified 



The Professional Education Division of the Air Force Human Resources 
Laboratory, mentioned in the Preface, was disbanded in July 1972 
and no longer exists as an organizational unit* 



SUMMARY 



Since 1968, Rand has been working on the design of instructional programs for 
the Air Force, especially for the Technical Schools in the Air Training Command. 
This report, one of a series of four, presents an overview of that work to date. To our 
knowledge this is the first attempt to devise a comprehensive and coherent me- 
thodology of instructional program design. 



BACKGROUND 

The Air Force expends several billion dollars annually on education and train- 
"ing; in technical training alone, it spends over half a billion dollars and graduates 
more than 150,000 men a year from five technical schools and numerous on-the-job 
training courses. The high cost of traditional methods of technical training has 
stimulated Air Force research on new educational methods. 

As in the public sector, until recently the process of instructional system design 
in the Air Force has still been guided largely by intuition, judgment, and an implicit 
acceptance of current operational procedures in the school and classroom. Many 
new teaching methods and instructional technologies have been developed and 
validated for their contributions to effective instruction. Their implementation has 
been slow and limited, however, mostly because it is so demanding a task to in- 
troduce change in established schools. 

The Air Force has been aware of the potential that these innovations offer and 
has instituted a process of instructional system development to help assure the 
releyance of subject matter in technical curricula. Rand's role has been to develop 
a similarly systematic process for designing the mix of instructors, facilities, materi- 
als, and students, and the processes by which all of these elements will work together 
to effect student mastery of the subject matter. 
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RESULTS 



As a result of this work, Rand has developed MODIA (A Method of Designing 
Instructional Alternatives), a comprehensive methodology for designing an instruc- 
tional system. MODIA consists of a sequence of procedures and semiautomated 
"tools" (some developed and some still in the planning stage) that allow a designer 
to examine many alternative structural approaches beforehe puts an actual system 
into use. He can rapidly plan a program for a particular approach and — stilt at the 
planning stage — assess its utility in terms of production of graduates or consump- 
tion of human and material resources. If the program is unacceptable on one of these 
counts, the designer may quickly construct and assess an alternative approach. 

The MODIA process consists of eight steps: 

1. Analyze characteristics of the learner population that will affect the way 
the course will be taught. 

2. Use the Questionnaire for Stating General Policy to specify the broad 
goals underlying the teaching institution's operation. 

3a. Use the Curriculum Analysis Questionnaire to typify each lesson in sys- 
tem-oriented terms and to characterize each lesson's requirements for com- 
munication media. 

3b. Specify instructional strategies with the help of DISTAF. Each strategy 
identifies the teaching agent and the way students v/ill interact with this 
teaching. ■ 

4. Specify design criteria input from the teaching institution, such as least 
course cost, shortest course length, or maximum student graduation rate. 

5. Describe local resources, such as the number of students entering the 
school and the school's resources and constraints. 

6. Design the instruction using as direct inputs information from the Cur- 
riculum Analysis Questionnaire, DISTAF, the design criteria, and the local 
resource description. 

7. Analyze the costs to determine the system's time-dependent dollar re- 
quirements. 

8. As necessary, depending on the acceptability of the outputs, repeat steps 
1-7 (with different inputs) until the most desirable system emerges. 

Some major tools must be developed before MODIA is complete, namely, a 
generalized computer model of student flow, a computer model for estimating re- 
source requirements, and a questionnaire for detailing local resources. Although 
work remains, MODIA was tested manually on part of a basic still photography 
course at Lowry Air Force Base, allowing us to draw three broad and important 
conclusions with confidence: 

• Systematic, generalized methods can be developed for designing programs 
of instruction; 
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These techniques encourage the examination of alternative approaches to 
instruction; ) 
A comprehensive, systematic approach stimulates the development of in- 
sights into the design of instruction that would otherwise not occur. 
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AN OVERVIEW OF MODIA: A METHOD OF DESIGNING 
INSTRUCTIONAL ALTERNATIVES FOR 
AIR FORCE TRAINING 



INTRODUCTION 

This report is an overview of the direction and scope of Rand work on instruc- 
tional program design. Readers who wish to pursue the subject in more depth should 
refer to the detailed reports cited in the Preface and at several points below. This 
series of reports will make results to date more accessible to people involved with 
the theory and practice of instructional program design and will provide a vehicle 
for critical comment. 



THE PROBLEM 

Annual Air Force expenditures vn education and training are on the order of 
several billion dollars. In technical training alone, the Air Force spends over half 
a billion dollars and graduates more than 150,000 men a year from five technical 
schools and numerous on-the-job training courses. Consequently, improvements in 
teaching effectiveness or savings in instructional costs can provide the Air Force 
with far greater benefits than those for individual students and classes/ 

As in the public sector, until recently the process of instructional program 
design in the Air Force has still been guided largely by intuition, judgment, and an 
implicit acceptance of current procedures of operation in the school and classroom. 
Many new teaching methods and instructional technologies have been developed 
and validated for their contributions to effective instruction. Some of these use 
material and personnel resources more eflficiently, some improve the quality of the 
instruction, some shorten training times. But their implementation has been slow 

^ An analysis of a similar magnifying effect of changes in the pilot training and retention process may 
— be-found in W/A. Stewart, Pilot Management Policy and Pilot Training Rates, The Rand Corporation, 
R-690-PR, March 1971. . 
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and limited, mostly because it is a demanding task to introduce change in estab- 
lished schools. 

'Because instruction is n ' x p»*'>'v^ss, those who want to change even a few 
of its aspects should ider^' litude of variables, most of whose interre- 

lations are difficult to iv< to unravel these complexities, most de- 

signers proposing cha: '^^T i simply assume that other aspects of h: Mic- 

tion will behave as usual, tet even this approach can involve the designer in a 
protracted exercise. More effective or less expensive instruction may result, but 
there is no assurance (1) that the resources required will not overtax the school; (2) 
that other changes could not have been made along with the changes of primary 
interest; and (3) that the designer has not overlooked more desirable combinations 
of techniques, operating procedures, personnel, materials, and equipment. 

The Air Force has been aware of the potential that the new teaching innova- 
tions offer and has instituted a process of instructional system development to help 
assure the relevance of subject matter in technical curricula. Rand's role has been 
to develop a "similarly systematic process for determining the mix of instructors, 
facilities, materials, and students and the processes by which all of these elements 
will work together to effect student mastery of the subject matter.^ The purpose of 
the work described here is to help implement changes in the instructional process 
in actual school situations through a process called MODI A (A Method of Designing 
Instructional Alternatives). MODIA consists of a sequence of procedures and 
semiautomated "tools," some of which have already been designed and are described 
in this series of reports.^ 



MAJOR BENEFITS OF MODIA 

We use the term tools because every attempt has been made to avoid prescribing 
whois to be taught a;Aa^ and Aoa;. Instead, given that the designer has established 
or been given general policies on these matters, these tools should help him put his 
policies into operation. Each tool performs tasks that can enhance the effectiveness 
of the others in achieving the final goal. At the same time, each one can be useful 
in its own right because it makes explicit various features of the instructional 
process that direct teaching in the classroom or workshop. 

^ We have emphasized technical training in the Air Force in developing the design methodology, and 
the detailed example described in the other reports draws from that area. The general structure of the 
methodology will also be useful, however, in other educational settings such as elementary and secondary 
education as well as civilian vocational training. Some revisions in details (particularly in the question- 
naire eliciting the Statement of General Policy) will be necessary for these latter applications. 

^ Users of such tools should be those responsible for selecting, designing, and implementing new 
instructional programs. Such people are found in the Training Research Applications Branches of the 
Technical Schools in the Air Training Command, in the administrative offices of large school districts, 
and in the teams that develop instructional programs for firms in the education industry. As the tools 
are more fully developed, they will enable these people to become more effective in shaping instruction 
to the needs of schools and students. 
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Probably MODIA's most important single contribution is that it encourages the 
examination of alternative approaches to instruction. The designer can rapidly plan 
a program for a particular approach, and — still at the planning stage— assess its 
utility in terms of production of graduates or consumption of human and material 
resources. If the resulting instructional program is unacceptable on one of these 
counts, the designer may quickly construct and assess an alternative approach. 
Providing this kind of feedback will help assure that many promising approaches 
are tried out, so to speak, before one is chosen for actual implementation. 

Another valuable contribution is that the method requires the designer to ar- 
ticulate his judgment of what constitutes effective instruction and to translate it into 
specific terms that direct the design in an orderly fashion. In this way, the prjfes- 
sional experience of the designer (or an educator working with him) becomes an 
integral part of the design process. Thus, even though methodology for predicting 
the specific outcomes of instruction in terms of student learning is currently lacking, 
judgment on this matter is an integral part of the methodology. In this way we hope 
to enlist the designer's support for the program he designs. 

In addition, MODIA encourages the designer to consider alternatives at many 
points in the design process, not. simply at the end of one design cycle. This exposes 
him to many options (most of which he may have been unaware of at the outset), 
thereby greatly increasing the variety of approaches he may wish to consider. At the 
same time, he may be encouraged to challenge some of the conventional methods 
that he has tacitly assumed apply. 

Finally, MODIA is comprehensive, in that it brings into explicit consideration 
all aspects of instruction that affect the instructional program; and it is coherent, 
in that wherever possible the interrelations of various aspects of the instructional 
process have been traced out. By means of MODIA, planners should be able to design 
an instructional program eflSciently and with confidence that its features are accept- 
able to the teaching institution and that it is appropriate to the learners. 



THE GENERAL APPROACH 

The remainder of this report describes our overall process of instructional pro- 
gram design. Throughout, it is assumed that the learning obj actives of the course 
are given, so that MODIA will complement current Air Force interest in systematiz- 
ing instruction as described in Air Force Manual Instructional System Develop- 
ment The manual describes a series of interrelated steps leading to the conduct and 
evaluation of instruction as illustrated in Fig. 1. After the requirements for system 
operation have been analyzed in terms of numbers of personnel with defined job 
skills (step 1), the training requirements are defined (step 2) and the objectives of 
instruction are developed (step 3). At this point (step 4), actual instruction must be 
planned, developed, and validated. This is the step the Rand team has been working 
on. 

MODIA is described in terms of its inputs, outputs, major components, and their 
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Fig. 1 — Steps in instructional system development, 
from Air Force Manual 50-2 
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Fig. 2 — Framework of design process 
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interrelations (Fig. 2). We have made some progress in identifying the steps in the 
design process, determining their sequence, and discovering their interrelation- 
ships. We plan to automate parts of MODIA so that it will take a matter of a few 
weeks or perhaps even a few days to design an instructional program, rather than 
several months to a year as it does at present 

Step 1 — Analy<::e the learner population in terms that will affect the v/ay the 
course will be taught (Fig. 3). For example, some learners may alieady have experi- 
ence in the particular field in which they will be studying. If the percentage of such 
learners va^ ' nth the time of year, this fact will also be included in the analysis. 
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State 
General 
Policy 
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• Course Length 
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• Time- Dependent 
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Fig. 3— First steps 



Step 2 — State general policy (Fig: 3) by means of a short questionnaire described 
by Carpenter.'* Policy, as used here, concerns three general areas. The first is the 



* Polly Carpenter, The MODIA Decision Process for Developing Strategies of Air Force Instruction, The 
Rand Corporation, R-1019-PR, November 1972. 
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broad characterization of the teaching institution's objectives. Many institutions 
have input-oriented objectives; some have output-oriented objectives. An input-ori- 
ented institution might be, for example, a labor union that specifies a certain num- 
ber of weeks in a vocational course as a membership requirement. Most military and 
industrial training, however, is output-oriented. Industry and the military want 
men with particular skills and knowledge; if it is possible to reduce the input 
required to get the same output, so much the better. 

The second policy area is the extent to which the school will adjust to variations 
in the student population. Does it want a standard or a diverse output? If the 
learners are fairly homogeneous, this question is not very important, but if they are 
heterogene^ Uii, it is. The Air Training Command tries to produce standard graduates 
in the technical center, although no one really believes that all airmen are identi- 
cally skilled when they have finished a technical course. The third policy area 
concerns the way in which the school must accommodate its operations to the needs 
of organizations that supply its students or accept its graduates. 

Step 3a— Analyze the curriculum using a branching questionnaire that guides 
the user in describing his course of study in detail (Fig. 4). Bretz describes this 
Curriculum Analysis Questionnaire.^ It typifies each lesson in system-oriented 
terms, such as whether the instruction must be given in a classroom or in a labora-^ 
tory, whether it requires special equipment, and whether it requires a monitor to 
ensure student safety. For example, if the students are learning how to take good 
pictures, the instruction requires special equipment; namely, a camera. If the stu- 
dents are learning only the theory of operating a camera, the instruction might take 
place entirely in a classroom with only visual aids to show how the camera operates. 

The Curriculum Analysis characterizes each lesson's requirements for com- 
munication media. We have focused on communication media for two reasons. First, 
they are becoming increasingly important in education. Second, many people are 
unfamiliar with media and their uses and are looking for guidance, which we believe 
we can provide.^ However, the demgn methodology does no^ specify that communica- 
tion mgdia must be used for every topic even though the Curriculum Analysis 
describes possible requirements for communication media for every topic. 

Step 3b~Specify strategies of instruction (Fig. .4). The framework for this specifi- 
cation is provided by the Statement of General Policy as well as by indirect input 
from the teaching institution. Step 3b interacts closely with the Curriculum Analy- 
sis. : 

As described, by Carpenter,*' Step 3b is a computer-directed logic tree called 
DISTAF (Determining Instructional Strategies for Training in the Air Force), with 
each decision point a logical consequence of the preceding decisions. To exercise it, 
the designer will have two aids: an interactive computer program written in JOSS, 

^ Rudy Bretz, The MODIA Questionnaire for Curriculum A nalysis,The Rand Corporat^^ 
November 1972. 

^ Rudy Bretz, The Selection of Appropriate Communication Media for Instruction: A Guide for Design- 
ers of Air Force Technical Training Programs, The Rand Corporation, R-601-PR, February 1971. 
Carpenter, op. cit. 
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Fig. 4 — Setting the stage 



and a manual that presents the pros and cons of the decisions to be made at each 
pointj along with some of the logical consequences of each choice.^ 

In Step 3b, specifications of instructional methods are explicitly entered into the 
design process/ DISTAF encourages the planner to consider alternative methods of 
instruction (up to 57 different teaching strategies for a single course) and to translate 
his decisions into guidehnes for program design. This allows him to take account of 
different levels of subject matter difficulty, different student abilities, and require- 
ments for differenttj^es of student performance. Thus, the process is prescrip- 
tive, because it champions no particular instructional method. Rather, it allows the 
planner to select and apply any method he may think appropriate, from the formal 
lecture to student-directed role-playing/ 

As used here, a strategy of instruction has two dimensions. For each type of 
instruction identified in the Curriculum Analysis, each strategy identifies (1) the 
teaching agent, and (2) the way students wiinnteract with this teaching. Carpenter 
discusses how answers to these two questions specify a teaching method. The 

® JOSS is Rand's on-line/interactive computer system. The JOSS program will be published as a 
' Supplement to R-1019-PR, op. cit., and eventually will be translated into Interactive FORTRAN. The 
manual appears as an appendix to R-1019-PR 
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strategy also permits specification of details of the use of media or personnel for each 
type of instruction, such as the level of skill the personnel should have. 

Step 4 — Specify design criteria input from the teaching institution (Fig. 4). They 
will be of the following sort: least cost, shortest course length, graduation of the most 
students per unit time, or maximum use of communication media. The designer 
would assign each criterion an order of importance or a weight. For example, be- 
cause of the value of student time to the Air Force, shortening the average time a 
student spends in a course might be weighted more heavily than saving on instruc- 
tors and training equipment. It is probable that such decisions should 1: nv^d\^ -^v 
school personnel. 

Step 5 — Gather descriptions of local resources and constraints by means of a 
logically structured set of questions (Fig. 5). These data will include the number of 
classrooms, laboratories, or other facilities available; the existence of communica- 
tion equipment such as television receivers installed in the classrooms; the rate of 
student entry; the geographic distribution of students, that is, whether they are all 
in one building or scattered throughout a campus or city; and the number of instruc- 
tors available. The resulting description of local resources and student loads will be 
used in the final design process to specify class size, select specific media systems, 
and serve other purposes. 

Step & — Design the instructional system using as direct inputs data from the 
Curriculum Analysis,;DISTAF, the design criteria, and the description of local 
resources (Fig. 5). Indirect inputs are characteristics of the learner population, the 
stated general policy, and the general course features. 

At pr-osent, Petruschell and Carpenter^ see the design process as having four 
main cjrioponents. First, each lesson is linked to the strategy of instruction that has 
been chosen for that particular category of instruction. Second, a set of criteria is 
used to select specific media and facilities. Third, a set of criteria is used to assign 
personnel. Sometimes a certain number of people will be required to carry out a 
particular task, such as monitoring for safety; other personnel requirements will 
depend on existing facilities^ such as the number of students that can feasibly be 
assigned to a teacher in a classroom. Fourth, student flow through the course is 
simulated by a flow and scheduling model to generate graduation rates and resource 
requirements. 

Step 7— Analyze the costs to determine the system's time-dependent dollar re- 
quirements (Fig. 5). This will eventually be accomplished by a computer program. 

Step 5— If the designer does not find the outputs of his planned strategy accepta- 
ble, he may repeat steps 1-7 (with different inputs) until the most desirable system 
emerges (Fig. 5). For instance, he can compare the, resource requirements with the 
resources he expects the school to have to determine whether the system is economi- 
cally feasible; he can also compare the outputs v.dth requirements for general policy 
and other inputs to determine whether they are what he wanted. If not, he can 

® The design process and an example of its application are described by R. L. Petruschell and Polly 
Carpenter in MODIA Applied in the Design and Cost Analysis of an Innovative Air Force Course, The 
Rand Corporation, R-1021-PR, November 1972. 
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!Fig- 5 — Tlie designi process 



change some of theiiiiitiaLsjpecifications; such as the strategies of instruction or the 
design criteria- Pos^bly, he would wantttio change the learner population, the course 
of study, or even tke geaeral policy. 

To denionstrate the feasibility and i utility of the design process, we manually 
worked through an exam^lfe based on an actual course in still photography taught 
to enlisted airmen in the Il&chnical School atliowry Air Force Base. For the exam- 
ple, we shortened the course, reducing: the. amount of data to be treated without 
losing the essential features^of the smbject matter in terms of variety of instruction 
and diversity of instructioEbsequencing. We;also revised the teaching method from 
the usual Air Force practiceito a more individualized approach that is of particular 
cerrent interest to the AirSrce. We then designed a paper program, using MODIA, 

estimated its requiremeimts for dollars anil other resources. An integral part 
rfthis exercise was the de\f€lopment of a model of the flow of students through the 
course. The resulting re(|mrements;;forTesouirces suggested that minor changes in 
instructional methods amfliSesign criteria might result in a significantly less expen- 
sive course that would largely retain the feature of individualization. 

*° These final steps are descmbed by Petruschell andl€arpenter in R-1021-PR. 



CONCLUSIONS 



There are still some major tools to be developed before MODIA is complete, 
namely, a generalized coms.ater model of student flow, a computer model for es- 
timating resource requirements, and a questionnaire for detailing existing local 
resources. Even so, we have reached the point where we can draw three broad and 
important conclusions with confidence: 

• Systematic, generalized methods can be developed for designing programs 
of instruction; 

• These techniques encourage the examination of alternative approaches to 
instruction; 

« A comprehensive, systematic approach stimulates the development of in- 
sights into the design of instruction that would not otherwise occur. 
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